Explanation of Parsing vs. Compilation vs. Runtime #8744
Conversation
dbeatty10
force-pushed
the
dbeatty/parsing-compilation-etc
branch
from
f83a15a to
7d73d81
Compare
Codecov ReportAll modified lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #8744 +/- ##
==========================================
- Coverage 86.65% 86.43% -0.23%
==========================================
Files 176 176
Lines 25772 26009 +237
==========================================
+ Hits 22332 22480 +148
- Misses 3440 3529 +89
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Thanks for making this happen @dbeatty10! Talk is cheap, PRs are precious ;)
- I left a first round of comments, mostly catching things that have changed since I initially wrote this last December
- As far as where in the repo this should land - I wonder if the
docs/architecturesubfolder could be the right place? Perhaps (in the future) accompanied by some diagrams?
Co-authored-by: Jeremy Cohen <[email protected]>
When ELI5 just isnt detailed enough.
I'm guessing we'll want to move and/or rename this at least one more time. But in the interim, moved it here:
|
|
|
||
| ### Parsing | ||
|
|
||
| As a user, you write models as SQL + YAML. dbt wants to understand each model as a Python object, defined by an internal data structure. It also wants to know its dependencies and configuration (= its place in the DAG). dbt reads your code **for that one model,** and attempts to construct that object, raising a **validation** error if it can’t. |
There was a problem hiding this comment.
SQL (or Python!) + YAML
I can see how that can make the rest of the section a little more complicated to follow, so feel free to keep it simple
There was a problem hiding this comment.
Good call!
Attempted to address this in 4fc26eb
Disambiguate Python references and delineate SQL models ("Jinja-SQL") from Python models ("dbt-py")
🎩 Preview
Problem
Solution
So let's publish something that we can use as a reference! Markdown within the code base would be a good first step.
Checklist
I have run this code in development and it appears to resolve the stated issueThis PR includes type annotations for new and modified functions